an interdisciplinary field that develops methods and software tools for understanding biological data, in particular when the data sets are large and complex
Other related applications
Data Explosion
Genetic Code
Every cell in our body contains DNA code
The code is written using the “letters” A, T, C, G
A blueprint for life (an analogy)
Imagine we are trying to construct a piece of furniture
We are often given a set of instructions to follow
A blueprint for life (an analogy)
Good, clear error-free instructions will give us the results we want
A blueprint for life (an analogy)
But errors in the instructions will produce a different result to expected
Some errors are more serious than others
Back to the the human genome
The code for a human is more like an entire bookcase
Errors can occur on any page, in any book, on any shelf
Why use computers
Nobody has the time and patience to search all these pages for errors
Computers never get bored, or need a rest
Computers are good at repetitive tasks and following instructions
However, computers still need humans to give them instructions (program)
and to fix them when errors occur
and to interpret the results
Demo time!
Data Management
Context
Context
What we actually spent time looking at (Disclaimer: not actual data from this project)
Context
No matter how much of the analysis is automated, some manual steps are inevitably involved
“metadata”
Sequencing technologies will happily generate the data required to assess what mutations are present in an individual
But they have no concept of the characteristics of that indiviual
patient / control
age
gender
drug treatment
Short Exercise
Imagine you have been asked to fill out the following spreadsheet for a clinical study involving smokers or non-smokers
What words, letters, etc would you put in the second column to indicate if the patient was a smoker or not?
Rule 1
Rule 1 - Maintain consistency
Example 1
Patient ID
Sex
Date of Diagnosis
Tumour Size
1
M
01-01-2013
3.1
2
f
04-18-1998
1.5
3
Male
1st of April 2004
105
4
Female
NA
67
5
F
2010/03/12
4.2
6
F
3.6
7
M
1994-11-05T08:15:30-05:00
232
Example 1
Consistency: F, female, f, fem, 2, …
Units
cm or mm; days, months or years
You can introduce inconsistencies without realising it
blank spaces (whitespace) at the end of text
“Male” is not the same as “Male”
Document choices you make about units in a README file
Regarding dates
credit: @myusuf3
Example 1 - corrected
Patient ID
Sex
Date of Diagnosis
Tumour Size
001
M
2013-01-01
3.1
002
F
1998-04-18
1.5
003
M
2004-04-01
1.05
004
F
NA
0.67
005
F
2010-03-12
4.2
006
F
NA
3.6
007
M
1994-11-05
2.32
Rule 2
Rule 2 - Never work directly on the raw data
Rule 2
Never work directly on the raw data
Hard to reverse all the manual steps performed and invites errors
Store the original data somewhere safe
see later
Rule 3
Figure showing locations of visitors to my Prostate Cancer data portal
Rule 3 - Don’t use 0 to mean missing
Zero values are data!
Sometimes extreme values such as 999 are sometimes used
NA is Ok, but what if NA is a valid category in your data?
R will recognise NA as a missing value and can ignore it in calculations
Safest to leave the cell empty
but you need to be careful with blank spaces
Rule 4
Patient ID
Date
Value
1
2015-06-14
213
2
76.5
3
2015-06-18
32
4
120.3
5
109
6
2015-06-20
7
143
Rule 4
Fill in all the cells
Rule 4
It is tempting to make the table look cleaner by not repeating some values
Fill in all cells!
otherwise, problems when sorting
Empty cell:
missing value?
value meant to be repeated multiple times?
Make sure it’s clear that the data is missing and not unintentionally left blank
Example 2 Corrected
Patient ID
Date
Value
1
2015-06-14
213
2
2015-06-14
76.5
3
2015-06-18
32
4
2015-06-18
120.3
5
2015-06-18
109
6
2015-06-20
NA
7
2015-06-20
143
Rule 5
Rule 5
Make it rectangle
The computer expects a very rigid shape of data with rows and columns
